1. Technical Field
The invention relates to world wide area networking in a computer environment. More particularly, the invention relates to delivering content and managing traffic across a world wide area network in a computer environment.
2. Description of the Prior Art
The Internet is a world wide xe2x80x9csuper-networkxe2x80x9d which connects together millions of individual computer networks and computers. The Internet is generally not a single entity. It is an extremely diffuse and complex system over where no single entity has complete authority or control. Although the Internet is widely know for one of its ways of presenting information through the World Wide We b (herein xe2x80x9cWebxe2x80x9d), there are many other services currently available based upon the general Internet protocols and infrastructure.
The Web is often easy to use for people inexperienced with computers. Information on the Web often is presented on xe2x80x9cpagesxe2x80x9d of graphics and text that contain xe2x80x9clinksxe2x80x9d to other pages either within the same set of data files (i.e., Web site) or within data files located on other computer networks. Users often access information on the Web using a xe2x80x9cbrowserxe2x80x9d program such as one made by Netscape Communications Corporation (now America Online, Inc.) of Mountain View, Calif. or Explorer(trademark) from Microsoft Corporation of Redmond, Wash. Browser programs can process information from Web sites and display the information using graphics, text, sound, and animation. Accordingly, the Web has become a popular medium for advertising goods and services directly to consumers.
As time progressed, usage of the Internet has exploded. There are literally millions of users on the Internet. Usage of the Internet is increasing daily and will eventually be in the billions of users. As usage increases so does traffic on the Internet. Traffic generally refers to the transfer of information from a Web site at a server computer to a user at a client computer. The traffic generally travels through the world wide network of computers using a packetized communication protocol, such as TCP/IP. Tiny packets of information travel from the server computer through the network to the client computer. Like automobiles during xe2x80x9crush hourxe2x80x9d on Highway 101 in Silicon Valley, the tiny packets of information traveling through the Internet become congested. Here, traffic jams which cause a delay in the information from the server to the client occur during high usage hours on the Internet. These traffic jams lead to long wait times at the client location. Here, a user of the client computer may wait for a long time for a graphical object to load onto his/her computer.
From the above, it is seen that an improved way to transfer information over a network is highly desirable.
It would be advantageous to provide content delivery and global traffic management network system that allows network traffic to be efficiently distributed to content servers by load balancing requests among servers and providing cached content for faster response times. It would further be advantageous to provide a content delivery and global traffic management network system that allows system administrators to collect traffic and server information.
The invention provides a content delivery and global traffic management network system. The system provides efficiently distributed network traffic to content servers by load balancing requests among servers and provides cached content for faster response times. In addition, the invention provides traffic and server information gathering for up to date system status.
A preferred embodiment of the invention provides a plurality of caching servers connected to a network. The caching servers host customer content that can be cached and stored, e.g., images, video, text, and/or software. The caching servers respond to requests for Web content from clients (Web browsers). If the requested content does not exist in memory or on disk, it generates a request to an origin site to obtain the content.
A Speedera DNS Server (SPD) load balances network requests among customer Web servers and directs client requests for hosted customer content to the appropriate caching server. The appropriate caching server is selected by choosing the caching server that is closest to the user, is available, and is the least loaded.
SPD also supports persistence. For persistent hostnames, SPD returns the same IP addresses, for a given client. The SPD server maintains a table containing the IP address given out for a given hostname to a client. This table is created dynamically in response to incoming requests and is synchronized across all the SPD servers responsible for a given zone. If the same client tries to resolve the hostname against a different SPD server in the future, it will get the same result.
To reduce the memory requirements and network traffic, the entire Internet address space is broken up into multiple zones. Each zone is assigned to a group of SPD servers. If an SPD server gets a request from a client that is not in the zone assigned to that SPD server, it forwards the request to the SPD server assigned to that zone. The SPD servers need to keep latency and persistence information only for the clients that fall in the zone assigned to the server. The latency probes only send the client latency information back to the SPD servers responsible for that client. Also the SPD servers only need to synchronize the persistence table with the SPD servers responsible for that zone, not all the SPD servers in the network.
When SPD has to forward the DNS request to servers in another zone, it selects the server with the best (lowest) latency value. This allows the SPD server to dynamically load balance between the SPD servers in the same zone and avoid servers that may be down or are having some other problems. SPD supports a two-tier architecture that can be used to increase the number of DNS servers in the system to more than the maximum allowed for .com domains. It can also be used to direct the client DNS servers to the closet Speedera DNS servers and to prevent the client DNS server from flip-flopping between all the DNS servers authoritative for speedera.net domain.
A customer either delegates a DNS name to said DNS server using a CNAME or by directly delegating the domain or is assigned a host domain name that some portion of the customer""s Web site is associated with. When the host server for the assigned host domain name gets a hit for content it does not contain, it will hit the appropriate content on the customer""s Web site to pick up and store that content, otherwise, the host server fulfills the hit from its cache.
The caching servers make requests back to the origin customer server site at specified intervals to check to see if the content that the caching server has cached is fresh. Caching servers also look at expiry headers in the HTTP content it retrieves from the origin customer server site to ensure freshness.
A configuration file contains all the static information about the Speedera Network. It contains the list of POPS and the servers present at each POP. It also contains the list of hostnames serviced by the network and maps the hostnames to the servers that can serve the content for that hostname. Every server in the network that needs configuration information has a copy of the appropriate current configuration file. The configuration file that a server receives contains all the configuration information for the particular portion of the network that the server is responsible for. A system administrator can, at any time, push a new configuration file to all machines that need it.
The caching servers write information about the content delivered to log files that are picked up and maintained by the log server. At a regular fixed interval, the caching server compresses and sends the logs of the content delivered to the log analysis servers. The log server collects the log files and delivers them to a database server that extracts and correlates the information. The information is used for billing as well as by customers for log analysis.
Other aspects and advantages of the invention will become apparent from the following detailed description in combination with the accompanying drawings, illustrating, by way of example, the principles of the invention.